Goto

Collaborating Authors

 Fort Walton Beach



Decouple knowledge from parameters for plug-and-play language modeling

Cheng, Xin, Lin, Yankai, Chen, Xiuying, Zhao, Dongyan, Yan, Rui

arXiv.org Artificial Intelligence

Pre-trained language models(PLM) have made impressive results in various NLP tasks. It has been revealed that one of the key factors to their success is the parameters of these models implicitly learn all kinds of knowledge during pre-training. However, encoding knowledge implicitly in the model parameters has two fundamental drawbacks. First, the knowledge is neither editable nor scalable once the model is trained, which is especially problematic in that knowledge is consistently evolving. Second, it lacks interpretability and prevents humans from understanding which knowledge PLM requires for a certain problem. In this paper, we introduce PlugLM, a pre-training model with differentiable plug-in memory(DPM). The key intuition is to decouple the knowledge storage from model parameters with an editable and scalable key-value memory and leverage knowledge in an explainable manner by knowledge retrieval in the DPM. To justify this design choice, we conduct evaluations in three settings including: (1) domain adaptation. PlugLM obtains 3.95 F1 improvements across four domains on average without any in-domain pre-training. (2) knowledge update. PlugLM could absorb new knowledge in a training-free way after pre-training is done. (3) in-task knowledge learning. PlugLM could be further improved by incorporating training samples into DPM with knowledge prompting.


Driverless truck could take over dangerous job for road crews

AITopics Original Links

The crash trucks, fitted with a device called a truck-mounted attenuator, have been credited with saving lives. But the workers who drive them are inevitably placed in harm's way, "literally waiting to be struck," said Robert Roy, president of Royal Truck & Equipment Inc. in Coopersburg. On Monday, Royal demonstrated its new driverless crash truck that it hopes will some day improve safety at work zones around the country. Two of the autonomous vehicles will make their debut at highway construction sites in Florida by the end of the year under a state Department of Transportation demonstration program. "Any time a driver can be removed from these vehicles in a very dangerous situation, and if the vehicle's struck, there's nobody inside of it to receive the damage or the injuries, that's measuring success," Roy said.